Search Results: "christoph"

26 October 2016

Christoph Egger: Installing a python systemd service?

As web search engines and IRC seems to be of no help, maybe someone here has a helpful idea. I have some service written in python that comes with a .service file for systemd. I now want to build&install a working service file from the software's setup.py. I can override the build/build_py commands of setuptools, however that way I still lack knowledge wrt. the bindir/prefix where my service script will be installed. Solution Turns out, if you override the install command (not the install_data!), you will have self.root and self.install_scripts (and lots of other self.install_*). As a result, you can read the template and write the desired output file after calling super's run method. The fix was inspired by GateOne (which, however doesn't get the --root parameter right, you need to strip self.root from the beginning of the path to actually make that work as intended). As suggested on IRC, the snippet (and my software) no use pkg-config to get at the systemd path as well. This is a nice improvement orthogonal to the original problem. The implementation here follows bley.

def systemd_unit_path():
    try:
        command = ["pkg-config", "--variable=systemdsystemunitdir", "systemd"]
        path = subprocess.check_output(command, stderr=subprocess.STDOUT)
        return path.decode().replace('\n', '')
    except (subprocess.CalledProcessError, OSError):
        return "/lib/systemd/system"
class my_install(install):
    _servicefiles = [
        'foo/bar.service',
        ]
    def run(self):
        install.run(self)
        if not self.dry_run:
            bindir = self.install_scripts
            if bindir.startswith(self.root):
                bindir = bindir[len(self.root):]
            systemddir = "%s%s" % (self.root, systemd_unit_path())
            for servicefile in self._servicefiles:
                service = os.path.split(servicefile)[1]
                self.announce("Creating %s" % os.path.join(systemddir, service),
                              level=2)
                with open(servicefile) as servicefd:
                    servicedata = servicefd.read()
                with open(os.path.join(systemddir, service), "w") as servicefd:
                    servicefd.write(servicedata.replace("%BINDIR%", bindir))
Comments, suggestions and improvements, of course, welcome!

22 October 2016

Christoph Egger: Running Debian on the ClearFog

Back in August, I was looking for a Homeserver replacement. During FrOSCon I was then reminded of the Turris Omnia project by NIC.cz. The basic SoC (Marvel Armada 38x) seemed to be nice hand have decent mainline support (and, with the turris, users interested in keeping it working). Only I don't want any WIFI and I wasn't sure the standard case would be all that usefully. Fortunately, there's also a simple board available with the same SoC called ClearFog and so I got one of these (the Base version). With shipping and the SSD (the only 2242 M.2 SSD with 250 GiB I could find, a ADATA SP600) it slightly exceeds the budget but well. ClearFog with SSD When installing the machine, the obvious goal was to use mainline FOSS components only if possible. Fortunately there's mainline kernel support for the device as well as mainline U-Boot. First attempts to boot from a micro SD card did not work out at all, both with mainline U-Boot and the vendor version though. Turns out the eMMC version of the board does not support any micro SD cards at all, a fact that is documented but others failed to notice as well. U-Boot As the board does not come with any loader on eMMC and booting directly from M.2 requires removing some resistors from the board, the easiest way is using UART for booting. The vendor wiki has some shell script wrapping an included C fragment to feed U-Boot to the device but all that is really needed is U-Boot's kwboot utility. For some reason the SPL didn't properly detect UART booting on my device (wrong magic number) but patching the if (in arch-mvebu's spl.c) and always assume UART boot is an easy way around. The plan then was to boot a Debian armhf rootfs with a defconfig kernel from USB stick. and install U-Boot and the rootfs to eMMC from within that system. Unfortunately U-Boot seems to be unable to talk to the USB3 port so no kernel loading from there. One could probably make UART loading work but switching between screen for serial console and xmodem seemed somewhat fragile and I never got it working. However ethernet can be made to work, though you need to set eth1addr to eth3addr (or just the right one of these) in U-Boot, saveenv and reboot. After that TFTP works (but is somewhat slow). eMMC There's one last step required to allow U-Boot and Linux to access the eMMC. eMMC is wired to the same PINs as the SD card would be. However the SD card has an additional indicator pin showing whether a card is present. You might be lucky inserting a dummy card into the slot or go the clean route and remove the pin specification from the device tree.
--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -306,7 +307,6 @@
                        sdhci@d8000  
                                bus-width = <4>;
-                               cd-gpios = <&gpio0 20 GPIO_ACTIVE_LOW>;
                                no-1-8-v;
                                pinctrl-0 = <&clearfog_sdhci_pins
                                             &clearfog_sdhci_cd_pins>;
Next Up is flashing the U-Boot to eMMC. This seems to work with the vendor U-Boot but proves to be tricky with mainline. The fun part boils down to the fact that the boot firmware reads the first block from eMMC, but the second from SD card. If you write the mainline U-Boot, which was written and tested for SD card, to eMMC the SPL will try to load the main U-Boot starting from it's second sector from flash -- obviously resulting in garbage. This one took me several tries to figure out and made me read most of the SPL code for the device. The fix however is trivial (apart from the question on how to support all different variants from one codebase, which I'll leave to the U-Boot developers):
--- a/include/configs/clearfog.h
+++ b/include/configs/clearfog.h
@@ -143,8 +143,7 @@
 #define CONFIG_SPL_LIBDISK_SUPPORT
 #define CONFIG_SYS_MMC_U_BOOT_OFFS             (160 << 10)
 #define CONFIG_SYS_U_BOOT_OFFS                 CONFIG_SYS_MMC_U_BOOT_OFFS
-#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        ((CONFIG_SYS_U_BOOT_OFFS / 512)\
-                                                + 1)
+#define CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_SECTOR        (CONFIG_SYS_U_BOOT_OFFS / 512)
 #define CONFIG_SYS_U_BOOT_MAX_SIZE_SECTORS     ((512 << 10) / 512) /* 512KiB */
 #ifdef CONFIG_SPL_BUILD
 #define CONFIG_FIXED_SDHCI_ALIGNED_BUFFER      0x00180000      /* in SDRAM */
Linux Now we have a System booting from eMMC with mainline U-Boot (which is a most welcome speedup compared to the UART and TFTP combination from the beginning). Getting to fine-tune linux on the device -- we want to install the armmp Debian kernel and have it work. As all the drivers are build as modules for that kernel this also means initrd support. Funnily U-Boots bootz allows booting a plain vmlinux kernel but I couldn't get it to boot a plain initrd. Passing a uImage initrd and a normal kernel however works pretty well. Back when I first tried there were some modules missing and ethernet didn't work with the PHY driver built as a module. In the meantime the PHY problem was fixed in the Debian kernel and almost all modules already added. Ben then only added the USB3 module on my suggestion and as a result, unstable's armhf armmp kernel should work perfectly well on the device (you still need to patch the device tree similar to the patch above). Still missing is an updated flash-kernel to automatically generate the initrd uImage which is work in progress but got stalled until I fixed the U-Boot on eMMC problem and everything should be fine -- maybe get debian u-boot builds for that board. Pro versus Base The main difference so far between the Pro and the Base version of the ClearFog is the switch chip which is included on the Pro. The Base instead "just" has two gigabit ethernet ports and a SFP. Both, linux' and U-Boot's device tree are intended for the Pro version which makes on of the ethernet ports unusable (it tries to find the switch behind the ethernet port which isn't there). To get both ports working (or the one you settled on earlier) there's a second patch to the device tree (my version might be sub-optimal but works), U-Boot -- the linux-kernel version is a trivial adaption:
--- a/arch/arm/dts/armada-388-clearfog.dts
+++ b/arch/arm/dts/armada-388-clearfog.dts
@@ -89,13 +89,10 @@
                internal-regs  
                        ethernet@30000  
                                mac-address = [00 50 43 02 02 02];
+                               managed = "in-band-status";
+                               phy = <&phy1>;
                                phy-mode = "sgmii";
                                status = "okay";
-
-                               fixed-link  
-                                       speed = <1000>;
-                                       full-duplex;
-                                ;
                         ;
                        ethernet@34000  
@@ -227,6 +224,10 @@
                                pinctrl-0 = <&mdio_pins>;
                                pinctrl-names = "default";
+                               phy1: ethernet-phy@1   /* Marvell 88E1512 */
+                                    reg = <1>;
+                                ;
+
                                phy_dedicated: ethernet-phy@0  
                                        /*
                                         * Annoyingly, the marvell phy driver
@@ -386,62 +386,6 @@
                tx-fault-gpio = <&expander0 13 GPIO_ACTIVE_HIGH>;
         ;
-       dsa@0  
-               compatible = "marvell,dsa";
-               dsa,ethernet = <&eth1>;
-               dsa,mii-bus = <&mdio>;
-               pinctrl-0 = <&clearfog_dsa0_clk_pins &clearfog_dsa0_pins>;
-               pinctrl-names = "default";
-               #address-cells = <2>;
-               #size-cells = <0>;
-
-               switch@0  
-                       #address-cells = <1>;
-                       #size-cells = <0>;
-                       reg = <4 0>;
-
-                       port@0  
-                               reg = <0>;
-                               label = "lan1";
-                        ;
-
-                       port@1  
-                               reg = <1>;
-                               label = "lan2";
-                        ;
-
-                       port@2  
-                               reg = <2>;
-                               label = "lan3";
-                        ;
-
-                       port@3  
-                               reg = <3>;
-                               label = "lan4";
-                        ;
-
-                       port@4  
-                               reg = <4>;
-                               label = "lan5";
-                        ;
-
-                       port@5  
-                               reg = <5>;
-                               label = "cpu";
-                        ;
-
-                       port@6  
-                               /* 88E1512 external phy */
-                               reg = <6>;
-                               label = "lan6";
-                               fixed-link  
-                                       speed = <1000>;
-                                       full-duplex;
-                                ;
-                        ;
-                ;
-        ;
-
        gpio-keys  
                compatible = "gpio-keys";
                pinctrl-0 = <&rear_button_pins>;
Conclusion Apart from the mess with eMMC this seems to be a pretty nice device. It's now happily running with a M.2 SSD providing enough storage for now and still has a mSATA/mPCIe plug left for future journeys. It seems to be drawing around 5.5 Watts with SSD and one Ethernet connected while mostly idle and can feed around 500 Mb/s from disk over an encrypted ethernet connection which is, I guess, not too bad. My plans now include helping to finish flash-kernel support, creating a nice case and probably get it deployed. I might bring it to FOSDEM first though. Working on it was really quite some fun (apart from the frustrating parts finding the one-block-offset ..) and people were really helpful. Big thanks here to Debian's arm folks, Ben Hutchings the kernel maintainer and U-Boot upstream (especially Tom Rini and Stefan Roese)

19 October 2016

Pau Garcia i Quiles: FOSDEM Desktops DevRoom 2017 all for Participation

FOSDEM is one of the largest (5,000+ hackers!) gatherings of Free Software contributors in the world and happens each February in Brussels (Belgium, Europe). Once again, one of the tracks will be the Desktops DevRoom (formerly known as CrossDesktop DevRoom ), which will host Desktop-related talks. We are now inviting proposals for talks about Free/Libre/Open-source Software on the topics of Desktop development, Desktop applications and interoperability amongst Desktop Environments. This is a unique opportunity to show novel ideas and developments to a wide technical audience. Topics accepted include, but are not limited to: Talks can be very specific, such as the advantages/disadvantages of distributing a desktop application with snap vs flatpak, or as general as using HTML5 technologies to develop native applications. Topics that are of interest to the users and developers of all desktop environments are especially welcome. The FOSDEM 2016 schedule might give you some inspiration. Submissions Please include the following information when submitting a proposal: How to submit All submissions are made in the Pentabarf event planning tool: https://penta.fosdem.org/submission/FOSDEM17 To submit your talk, click on Create Event , then make sure to select the Desktops devroom as the Track . Otherwise your talk will not be even considered for any devroom at all. If you already have a Pentabarf account from a previous year, even if your talk was not accepted, please reuse it. Create an account if, and only if, you don t have one from a previous year. If you have any issues with Pentabarf, please contact desktops-devroom@lists.fosdem.org. Deadline The deadline for submissions is December 5th 2016. FOSDEM will be held on the weekend of 4 & 5 February 2017 and the Desktops DevRoom will take place on Sunday, February 5th 2017. We will contact every submitter with a yes or no before December 11th 2016. Recording permission The talks in the Desktops DevRoom will be audio and video recorded, and possibly streamed live too. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example:
If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>.
If you want us to stop the recording in the Q & A part (should you have one), please tell us. We can do that but only for the Q & A part. More information The official communication channel for the Desktops DevRoom is its mailing list desktops-devroom@lists.fosdem.org. Use this page to manage your subscription: https://lists.fosdem.org/listinfo/desktops-devroom Organization The Desktops DevRoom 2017 is managed by a team representing the most notable open desktops: If you want to join the team, please contact desktops-devroom@lists.fosdem.org

Reproducible builds folks: Reproducible Builds: week 77 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday October 9 and Saturday October 15 2016: Media coverage Documentation update After discussions with HW42, Steven Chamberlain, Vagrant Cascadian, Daniel Shahaf, Christopher Berg, Daniel Kahn Gillmor and others, Ximin Luo has started writing up more concrete and detailed design plans for setting SOURCE_ROOT_DIR for reproducible debugging symbols, buildinfo security semantics and buildinfo security infrastructure. Toolchain development and fixes Dmitry Shachnev noted that our patch for #831779 has been temporarily rejected by docutils upstream; we are trying to persuade them again. Tony Mancill uploaded javatools/0.59 to unstable containing original patch by Chris Lamb. This fixed an issue where documentation Recommends: substvars would not be reproducible. Ximin Luo filed bug 77985 to GCC as a pre-requisite for future patches to make debugging symbols reproducible. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible - in our current test setup - after being fixed: The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) Some uploads have addressed some reproducibility issues, but not all of them: Some uploads have addressed nearly all reproducibility issues, except for build path issues: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 101 package reviews have been added, 49 have been updated and 4 have been removed in this week, adding to our knowledge about identified issues. 3 issue types have been updated: Weekly QA work During of reproducibility testing, some FTBFS bugs have been detected and reported by: tests.reproducible-builds.org Debian: Openwrt/LEDE/NetBSD/coreboot/Fedora/archlinux: Misc. We are running a poll to find a good time for an IRC meeting. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

6 October 2016

Reproducible builds folks: Reproducible Builds: week 75 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 25 and Saturday October 1 2016: Statistics For the first time, we reached 91% reproducible packages in our testing framework on testing/amd64 using a determistic build path. (This is what we recommend to make packages in Stretch reproducible.) For unstable/amd64, where we additionally test for reproducibility across different build paths we are at almost 76% again. IRC meetings We have a poll to set a time for a new regular IRC meeting. If you would like to attend, please input your available times and we will try to accommodate for you. There was a trial IRC meeting on Friday, 2016-09-31 1800 UTC. Unfortunately, we did not activate meetbot. Despite this participants consider the meeting a success as several topics where discussed (eg changes to IRC notifications of tests.r-b.o) and the meeting stayed within one our length. Upcoming events Reproduce and Verify Filesystems - Vincent Batts, Red Hat - Berlin (Germany), 5th October, 14:30 - 15:20 @ LinuxCon + ContainerCon Europe 2016. From Reproducible Debian builds to Reproducible OpenWrt, LEDE & coreboot - Holger "h01ger" Levsen and Alexander "lynxis" Couzens - Berlin (Germany), 13th October, 11:00 - 11:25 @ OpenWrt Summit 2016. Introduction to Reproducible Builds - Vagrant Cascadian will be presenting at the SeaGL.org Conference In Seattle (USA), November 11th-12th, 2016. Previous events GHC Determinism - Bartosz Nitka, Facebook - Nara (Japan), 24th September, ICPF 2016. Toolchain development and fixes Michael Meskes uploaded bsdmainutils/9.0.11 to unstable with a fix for #830259 based on Reiner Herrmann's patch. This fixed locale_dependent_symbol_order_by_lorder issue in the affected packages (freebsd-libs, mmh). devscripts/2.16.8 was uploaded to unstable. It includes a debrepro script by Antonio Terceiro which is similar in purpose to reprotest but more lightweight; specific to Debian packages and without support for virtual servers or configurable variations. Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible in our testing framework after being fixed: The following updated packages appear to be reproducible now for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 77 package reviews have been added, 178 have been updated and 80 have been removed in this week, adding to our knowledge about identified issues. 6 issue types have been updated: Weekly QA work As part of reproducibility testing, FTBFS bugs have been detected and reported by: diffoscope development A new version of diffoscope 61 was uploaded to unstable by Chris Lamb. It included contributions from: Post-release there were further contributions from: reprotest development A new version of reprotest 0.3.2 was uploaded to unstable by Ximin Luo. It included contributions from: Post-release there were further contributions from: tests.reproducible-builds.org Misc. This week's edition was written by Ximin Luo, Holger Levsen & Chris Lamb and reviewed by a bunch of Reproducible Builds folks on IRC.

26 September 2016

Kees Cook: security things in Linux v4.3

When I gave my State of the Kernel Self-Protection Project presentation at the 2016 Linux Security Summit, I included some slides covering some quick bullet points on things I found of interest in recent Linux kernel releases. Since there wasn t a lot of time to talk about them all, I figured I d make some short blog posts here about the stuff I was paying attention to, along with links to more information. This certainly isn t everything security-related or generally of interest, but they re the things I thought needed to be pointed out. If there s something security-related you think I should cover from v4.3, please mention it in the comments. I m sure I haven t caught everything. :) A note on timing and context: the momentum for starting the Kernel Self Protection Project got rolling well before it was officially announced on November 5th last year. To that end, I included stuff from v4.3 (which was developed in the months leading up to November) under the umbrella of the project, since the goals of KSPP aren t unique to the project nor must the goals be met by people that are explicitly participating in it. Additionally, not everything I think worth mentioning here technically falls under the kernel self-protection ideal anyway some things are just really interesting userspace-facing features. So, to that end, here are things I found interesting in v4.3: CONFIG_CPU_SW_DOMAIN_PAN Russell King implemented this feature for ARM which provides emulated segregation of user-space memory when running in kernel mode, by using the ARM Domain access control feature. This is similar to a combination of Privileged eXecute Never (PXN, in later ARMv7 CPUs) and Privileged Access Never (PAN, coming in future ARMv8.1 CPUs): the kernel cannot execute user-space memory, and cannot read/write user-space memory unless it was explicitly prepared to do so. This stops a huge set of common kernel exploitation methods, where either a malicious executable payload has been built in user-space memory and the kernel was redirected to run it, or where malicious data structures have been built in user-space memory and the kernel was tricked into dereferencing the memory, ultimately leading to a redirection of execution flow. This raises the bar for attackers since they can no longer trivially build code or structures in user-space where they control the memory layout, locations, etc. Instead, an attacker must find areas in kernel memory that are writable (and in the case of code, executable), where they can discover the location as well. For an attacker, there are vastly fewer places where this is possible in kernel memory as opposed to user-space memory. And as we continue to reduce the attack surface of the kernel, these opportunities will continue to shrink. While hardware support for this kind of segregation exists in s390 (natively separate memory spaces), ARM (PXN and PAN as mentioned above), and very recent x86 (SMEP since Ivy-Bridge, SMAP since Skylake), ARM is the first upstream architecture to provide this emulation for existing hardware. Everyone running ARMv7 CPUs with this kernel feature enabled suddenly gains the protection. Similar emulation protections (PAX_MEMORY_UDEREF) have been available in PaX/Grsecurity for a while, and I m delighted to see a form of this land in upstream finally. To test this kernel protection, the ACCESS_USERSPACE and EXEC_USERSPACE triggers for lkdtm have existed since Linux v3.13, when they were introduced in anticipation of the x86 SMEP and SMAP features. Ambient Capabilities Andy Lutomirski (with Christoph Lameter and Serge Hallyn) implemented a way for processes to pass capabilities across exec() in a sensible manner. Until Ambient Capabilities, any capabilities available to a process would only be passed to a child process if the new executable was correctly marked with filesystem capability bits. This turns out to be a real headache for anyone trying to build an even marginally complex least privilege execution environment. The case that Chrome OS ran into was having a network service daemon responsible for calling out to helper tools that would perform various networking operations. Keeping the daemon not running as root and retaining the needed capabilities in children required conflicting or crazy filesystem capabilities organized across all the binaries in the expected tree of privileged processes. (For example you may need to set filesystem capabilities on bash!) By being able to explicitly pass capabilities at runtime (instead of based on filesystem markings), this becomes much easier. For more details, the commit message is well-written, almost twice as long as than the code changes, and contains a test case. If that isn t enough, there is a self-test available in tools/testing/selftests/capabilities/ too. PowerPC and Tile support for seccomp filter Michael Ellerman added support for seccomp to PowerPC, and Chris Metcalf added support to Tile. As the seccomp maintainer, I get excited when an architecture adds support, so here we are with two. Also included were updates to the seccomp self-tests (in tools/testing/selftests/seccomp), to help make sure everything continues working correctly. That s it for v4.3. If I missed stuff you found interesting, please let me know! I m going to try to get more per-version posts out in time to catch up to v4.8, which appears to be tentatively scheduled for release this coming weekend.

2016, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.
Creative Commons License

20 September 2016

Reproducible builds folks: Reproducible Builds: week 73 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 11 and Saturday September 17 2016: Toolchain developments Ximin Luo started a new series of tools called (for now) debrepatch, to make it easier to automate checks that our old patches to Debian packages still apply to newer versions of those packages, and still make these reproducible. Ximin Luo updated one of our few remaining patches for dpkg in #787980 to make it cleaner and more minimal. The following tools were fixed to produce reproducible output: Packages reviewed and fixed, and bugs filed The following updated packages have become reproducible - in our current test setup - after being fixed: The following updated packages appear to be reproducible now, for reasons we were not able to figure out. (Relevant changelogs did not mention reproducible builds.) The following 3 packages were not changed, but have become reproducible due to changes in their build-dependencies: jaxrs-api python-lua zope-mysqlda. Some uploads have addressed some reproducibility issues, but not all of them: Patches submitted that have not made their way to the archive yet: Reviews of unreproducible packages 462 package reviews have been added, 524 have been updated and 166 have been removed in this week, adding to our knowledge about identified issues. 25 issue types have been updated: Weekly QA work FTBFS bugs have been reported by: diffoscope development A new version of diffoscope 60 was uploaded to unstable by Mattia Rizzolo. It included contributions from: It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 72 71 70. strip-nondeterminism development New versions of strip-nondeterminism 0.027-1 and 0.028-1 were uploaded to unstable by Chris Lamb. It included contributions from: disorderfs development A new version of disorderfs 0.5.1 was uploaded to unstable by Chris Lamb. It included contributions from: It also included from changes previous weeks; see either the changes or commits linked above, or previous blog posts 70. Misc. This week's edition was written by Ximin Luo and reviewed by a bunch of Reproducible Builds folks on IRC.

12 September 2016

Reproducible builds folks: Reproducible Builds: week 72 in Stretch cycle

What happened in the Reproducible Builds effort between Sunday September 4 and Saturday September 10 2016: Reproducible work in other projects Python 3.6's dictonary type now retains the insertion order. Thanks to themill for the report. In coreboot, Alexander Couzens committed a change to make their release archives reproducible. Patches submitted Reviews of unreproducible packages We've been adding to our knowledge about identified issues. 3 issue types have been added: 1 issue type has been updated: 16 have been have updated: 13 have been removed, not including removed packages: 100s of packages have been tagged with the more generic captures_build_path, and many with captures_kernel_version, user_hostname_manually_added_requiring_further_investigation, user_hostname_manually_added_requiring_further_investigation, captures_shell_variable_in_autofoo_script, etc. Particular thanks to Emanuel Bronshtein for his work here. Weekly QA work FTBFS bugs have been reported by: diffoscope development strip-nondeterminism development tests.reproducible-builds.org: Misc. This week's edition was written by Chris Lamb and Holger Levsen and reviewed by a bunch of Reproducible Builds folks on IRC.

10 September 2016

Sylvain Le Gall: Release of OASIS 0.4.7

I am happy to announce the release of OASIS v0.4.7. Logo OASIS small OASIS is a tool to help OCaml developers to integrate configure, build and install systems in their projects. It should help to create standard entry points in the source code build system, allowing external tools to analyse projects easily. This tool is freely inspired by Cabal which is the same kind of tool for Haskell. You can find the new release here and the changelog here. More information about OASIS in general on the OASIS website. Pull request for inclusion in OPAM is pending. Here is a quick summary of the important changes: Features: This version contains a lot of changes and is the achievement of a huge amount of work. The addition of OMake as a plugin is a huge progress. The overall work has been targeted at making OASIS more library like. This is still a work in progress but we made some clear improvement by getting rid of various side effect (like the requirement of using "chdir" to handle the "-C", which leads to propage ~ctxt everywhere and design OASISFileSystem). I would like to thanks again the contributor for this release: Spiros Eliopoulos, Paul Snively, Jeremie Dimino, Christopher Zimmermann, Christophe Troestler, Max Mouratov, Jacques-Pascal Deplaix, Geoff Shannon, Simon Cruanes, Vladimir Brankov, Gabriel Radanne, Evgenii Lepikhin, Petter Urkedal, Gerd Stolpmann and Anton Bachin.

30 August 2016

Christoph Egger: DANE and DNSSEC Monitoring

At this year's FrOSCon I repeted my presentation on DNSSEC. In the audience, there was the suggestion of a lack of proper monitoring plugins for a DANE and DNSSEC infrastructure that was easily available. As I already had some personal tools around and some spare time to burn I've just started a repository with some useful tools. It's available on my website and has mirrors on Gitlab and Github. I intent to keep this repository up-to-date with my personal requirements (which also means adding a xmpp check soon) and am happy to take any contributions (either by mail or as "pull requests" on one of the two mirrors). It currently has smtp (both ssmtp and starttls) and https support as well as support for checking valid DNSSEC configuration of a zone. While working on it it turned out some things can be complicated. My language of choice was python3 (if only because the ssl library has improved since 2.7 a lot), however ldns and unbound in Debian lack python3 support in their bindings. This seems fixable as the source in Debian is buildable and useable with python3 so it just needs packaging adjustments. Funnily the ldns module, which is only needed for check_dnssec, in debian is currently buggy for python2 and python3 and ldns' python3 support is somewhat lacking so I spent several hours hunting SWIG problems.

11 August 2016

Christoph Egger: Looking for a replacement Homeserver

Almost exactly six years ago I bought one of these Fuloong 6064 mini PCs. The machine has been working great ever since both collecting my mail and acting as an IMAP server as well as providing public services -- it's also keyserver.siccegge.de. However jessie is supposed to be the last Debian release supporting the hardware and the system's rather slow and lacks memory. This is especially noticeable with IMAP spam filter training and mail indexing. Therefore I'm looking for some nice replacement -- preferably non-x86 again (no technical reasons). My requirements are pretty simple: Now I'd consider one of these ARM boards and get it a nice case but they seem all to either fail in terms of SATA or not being faster at all (and one needs to go for outdated hardware to stand a chance of mainline kernel support). If anyone knows something nice and non-x86 I'll happily take suggestions.

29 May 2016

Christoph Berg: vcswatch is now looking for tags

About a week ago, I extended vcswatch to also look at tags in git repositories. Previously, it was solely paying attention to the version number in the top paragraph in debian/changelog, and would alert if that version didn't match the package version in Debian unstable or experimental. The idea is that "UNRELEASED" versions will keep nagging the maintainer (via DDPO) not to forget that some day this package needs an upload. This works for git, svn, bzr, hg, cvs, mtn, and darcs repositories (in decreasing order of actual usage numbers in Debian. I had actually tried to add arch support as well, but that VCS is so weird that it wasn't worth the trouble). There are several shortcomings in that simple approach: The new mechanism fixes this for git repositories by also looking at the output of git describe --tags. If there are any commits since the last tag, and the vcswatch status according to debian/changelog would otherwise be "OK", a new status "COMMITS" is set. DDPO will report e.g. "1.4-1+2", to be read as "2 commits since the tag [debian/]1.4-1". Of the 16644 packages using git in Debian, currently 7327 are "OK", 2649 are in the new "COMMITS" state, and 4227 are "NEW". 723 are "OLD" and 79 are "UNREL" which indicates that the package in Debian is ahead of the git repository. 1639 are in an ERROR state. So far the new mechanism works for git only, but other VCSes could be added as well.

18 April 2016

Reproducible builds folks: Reproducible builds: week 50 in Stretch cycle

What happened in the reproducible builds effort between April 3rd and April 9th 2016: Media coverage Emily Ratliff wrote an article for SecurityWeek called Establishing Correspondence Between an Application and its Source Code - How Combining Two Completely Separate Open Source Projects Can Make Us All More Secure. Tails have started work on a design for freezable APT repositories to make it easier and practical to perform reproductions of an entire distribution at a given point in time, which will be needed to create reproducible installation- or live-media. Toolchain fixes Alexis Bienven e submitted patches adding support for SOURCE_DATE_EPOCH in several tools: transfig, imagemagick, rdtool, and asciidoctor. boyska submitted one for python-reportlab. Packages fixed The following packages have become reproducible due to changes in their build dependencies: atinject-jsr330 brailleutils cglib3 gnugo libcobra-java libgnumail-java libjchart2d-java libjcommon-java libjfreechart-java libjide-oss-java liblaf-widget-java liblastfm-java liboptions-java octave-control octave-mpi octave-nan octave-parallel octave-stk octave-struct octave-tsa oar The following packages became reproducible after getting fixed: Several uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet: Other upstream fixes Alexander Batischev made a commit to make newsbeuter reproducible. tests.reproducible-builds.org Package reviews 93 reviews have been removed, 66 added and 21 updated in the previous week. 12 new FTBFS bugs have been reported by Chris Lamb and Niko Tyni. Misc. This week's edition was written by Lunar, Holger Levsen, Reiner Herrmann, Mattia Rizzolo and Ximin Luo. With the departure of Lunar as a full-time contributor, Reproducible Builds Weekly News (this thing you're reading) has moved from his personal Debian blog on Debian People to the Reproducible Builds team web site on Debian Alioth. You may want to update your RSS or Atom feeds. Very many thanks to Lunar for writing and publishing this weekly news for so long, well & continously!

14 March 2016

Lunar: Reproducible builds: week 46 in Stretch cycle

What happened in the reproducible builds effort between March 6th and March 12th:

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dfc, gap-openmath, gnubik, gplanarity, iirish, iitalian, monajat, openimageio, plexus-digest, ruby-fssm, vdr-plugin-dvd, vdr-plugin-spider. The following packages became reproducible after getting fixed:
  • adduser/3.114 by Niels Thykier.
  • bsdmainutils/9.0.7 by Michael Meskes.
  • criu/2.0-1 by Salvatore Bonaccorso.
  • genometools/1.5.8+ds-2 by Sascha Steinbiss.
  • gfs2-utils/3.1.8-1 uploaded by Bastian Blank, fix by Christoph Berg.
  • gmerlin/1.2.0~dfsg+1-5 by IOhannes m zm lnig.
  • heroes/0.21-14 by Stephen Kitt.
  • kmc/2.3+dfsg-3 by Sascha Steinbiss.
  • polyml/5.6-3 by James Clarke.
  • sed/4.2.2-7.1 by Niels Thykier.
  • snpomatic/1.0-3 by Sascha Steinbiss.
  • tantan/13-4 by Sascha Steinbiss.
Some uploads fixed some reproducibility issues, but not all of them: Patches submitted which have not made their way to the archive yet:
  • #817979 on modernizr by Sascha Steinbiss: sort list of files included in feature-detects.js.
  • #818027 on snapper by Sascha Steinbiss: always use /bin/sh as shell.

tests.reproducible-builds.org Always use all cores on armhf builders. (h01ger) Improve the look of Debian dashboard. (h01ger)

Package reviews 118 reviews have been removed, 114 added and 15 updated in the previous week. 15 FTBFS have been filled by Chris Lamb. New issues: xmlto_txt_output_locale_specific.

Misc. Lunar seeks new maintainers for diffoscope, several mailing lists, and these very weekly reports.

29 February 2016

Dirk Eddelbuettel: New CRAN package gunsales

This is based on joint work with Gregor Aisch and Josh Keller of the New York Times. A new package gunsales is now on the CRAN network for R. It is based the NYTimes/gunsales repository underlying the excellent New York Times visualizations, first published first in December 2015 and updated with more recent data since. The analysis takes public government data on gun sales from the National Instant Criminal Background Check System (NICS). The original data is scraped from the pdf, included in the package, and analysed in a cross-section and time-series manner. The standard US Census tool X-13ARIMA-SEATS is used to deseasonalize the timeseries at the national or state level. (Note that Buzzfeed also published data and (Python) code in another GitHub repo.) As an aside, it was the use of X-13ARIMA-SEATS here -- and its somewhat awkward and manual installation also seen in the initial versions of the code in the NYTimes/gunsales repo -- which lead to the recent work by Christoph Sax and myself. We now provide a new package x13binary on CRAN so that Christoph's excellent seasonal package can simply depend upon it and have a working binary provided and installed ready to use; see the recent blog post for more. The net result is that a package like this new gunsales project can simply depend upon seasonal and also be assurred that x13binary "just works". As Martha would say, "A Good Thing". Back to the gunsales project. Following the initial publication of the repository with the data and R code in a simple script, I felt compelled to reorganize it as a package. Packages for R, as we teach our students, colleagues, or anybody else who wants to listen are really the best way to bundle code, data, documentation (i.e. vignettes) and tests. All that exists now in the gunsales package. The package now has one main function, analysis(), which returns a single dataframe object. This dataframe object can then be fed to two plotting functions. The first, plot_gunsales(), will then recreate all the (base R) plots from the original code base. The second, ggplot_gunsales(), does the same but via ggplot2. This should give anybody the ability to look at the data, study the transformations done, form and maybe test new hypotheses and visualize in manner comparable to the original publication. As an amuse gueule, here are the key plots also shown in the main README.md at GitHub: Total Estimated Gun Sales Total Estimated Gun Sales, Seasonally Adjusted Total Estimated Gun Sales, Population-Growth Adjusted Handguns vs Longguns Six States DC We look forward to more remixes and analysis of this data. The plan of the GitHub repository is to keep the data set updated as new data points are published.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

24 February 2016

Christoph Egger: doveadm deduplicate

Without further words:
% for i in $(seq 1 90) ; do doveadm mailbox status messages debian.buildd.archive.2011.05   column -t ;  doveadm deduplicate mailbox debian.buildd.archive.2011.05 ; done
debian.buildd.archive.2011.05  messages=8094
debian.buildd.archive.2011.05  messages=7939
debian.buildd.archive.2011.05  messages=7816
debian.buildd.archive.2011.05  messages=7698
debian.buildd.archive.2011.05  messages=7610
debian.buildd.archive.2011.05  messages=7529
debian.buildd.archive.2011.05  messages=7455
debian.buildd.archive.2011.05  messages=7375
debian.buildd.archive.2011.05  messages=7294
debian.buildd.archive.2011.05  messages=7215
debian.buildd.archive.2011.05  messages=7136
debian.buildd.archive.2011.05  messages=7032
debian.buildd.archive.2011.05  messages=6941
debian.buildd.archive.2011.05  messages=6839
debian.buildd.archive.2011.05  messages=6721
debian.buildd.archive.2011.05  messages=6631
debian.buildd.archive.2011.05  messages=6553
debian.buildd.archive.2011.05  messages=6476
debian.buildd.archive.2011.05  messages=6388
debian.buildd.archive.2011.05  messages=6301
debian.buildd.archive.2011.05  messages=6211
debian.buildd.archive.2011.05  messages=6140
debian.buildd.archive.2011.05  messages=6056
debian.buildd.archive.2011.05  messages=6007
debian.buildd.archive.2011.05  messages=5955
debian.buildd.archive.2011.05  messages=5887
debian.buildd.archive.2011.05  messages=5826
debian.buildd.archive.2011.05  messages=5752
debian.buildd.archive.2011.05  messages=5706
debian.buildd.archive.2011.05  messages=5657
debian.buildd.archive.2011.05  messages=5612
debian.buildd.archive.2011.05  messages=5570
debian.buildd.archive.2011.05  messages=5523
debian.buildd.archive.2011.05  messages=5474
debian.buildd.archive.2011.05  messages=5422
debian.buildd.archive.2011.05  messages=5382
debian.buildd.archive.2011.05  messages=5343
debian.buildd.archive.2011.05  messages=5308
debian.buildd.archive.2011.05  messages=5256
debian.buildd.archive.2011.05  messages=5221
debian.buildd.archive.2011.05  messages=5168
debian.buildd.archive.2011.05  messages=5133
debian.buildd.archive.2011.05  messages=5092
debian.buildd.archive.2011.05  messages=5058
debian.buildd.archive.2011.05  messages=5030
debian.buildd.archive.2011.05  messages=4994
debian.buildd.archive.2011.05  messages=4964
debian.buildd.archive.2011.05  messages=4935
debian.buildd.archive.2011.05  messages=4900
debian.buildd.archive.2011.05  messages=4868
debian.buildd.archive.2011.05  messages=4838
debian.buildd.archive.2011.05  messages=4811
debian.buildd.archive.2011.05  messages=4778
debian.buildd.archive.2011.05  messages=4748
debian.buildd.archive.2011.05  messages=4722
debian.buildd.archive.2011.05  messages=4686
debian.buildd.archive.2011.05  messages=4661
debian.buildd.archive.2011.05  messages=4637
debian.buildd.archive.2011.05  messages=4613
debian.buildd.archive.2011.05  messages=4593
debian.buildd.archive.2011.05  messages=4570
debian.buildd.archive.2011.05  messages=4554
debian.buildd.archive.2011.05  messages=4536
debian.buildd.archive.2011.05  messages=4520
debian.buildd.archive.2011.05  messages=4500
debian.buildd.archive.2011.05  messages=4481
debian.buildd.archive.2011.05  messages=4466
debian.buildd.archive.2011.05  messages=4445
debian.buildd.archive.2011.05  messages=4430
debian.buildd.archive.2011.05  messages=4417
debian.buildd.archive.2011.05  messages=4405
debian.buildd.archive.2011.05  messages=4390
debian.buildd.archive.2011.05  messages=4376
debian.buildd.archive.2011.05  messages=4366
debian.buildd.archive.2011.05  messages=4360
debian.buildd.archive.2011.05  messages=4350
debian.buildd.archive.2011.05  messages=4336
debian.buildd.archive.2011.05  messages=4329
debian.buildd.archive.2011.05  messages=4320
debian.buildd.archive.2011.05  messages=4315
debian.buildd.archive.2011.05  messages=4312
debian.buildd.archive.2011.05  messages=4311
debian.buildd.archive.2011.05  messages=4309
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308
debian.buildd.archive.2011.05  messages=4308

23 February 2016

Uwe Kleine-K nig: Christoph Hellwig vs VMware

On Thursday the lawsuit about GPL compliance of VMware's ESXi products starts. I support Christoph's position and share his view that VMware's product is in conflict with the GPL. Read more on the software freedom conservancy's FAQ about the case. I keep my fingers crossed!

21 February 2016

Lunar: Reproducible builds: week 43 in Stretch cycle

What happened in the reproducible builds effort between February 14th and February 20th 2016:

Toolchain fixes Yaroslav Halchenko uploaded cython/0.23.4+git4-g7eed8d8-1 which makes its output deterministic. Original patch by Chris Lamb. Didier Raboud uploaded pyppd/1.0.2-3 to experimental which now serialize PPD deterministically. Lunar submitted two patches for lcms to add a way for clients to set the creation date/time in profile headers and initialize all bytes when writing named colors.

Packages fixed The following packages have become reproducible due to changes in their build dependencies: dbconfig-common, dctrl-tools, dvdwizard, ekg2, expeyes, galternatives, gpodder, icewm, latex-mk, libiio, lives, navit, po4a, tasksel, tilda, vdr-plugin-infosatepg, xaos. The following packages became reproducible after getting fixed: Some uploads fixed some reproducibility issues, but not all of them: Unknown status:
  • tomcat7/7.0.68-1 by Emmanuel Bourg (test suite fails in test environment).
Patches submitted which have not made their way to the archive yet:
  • #814840 on tor by Petter Reinholdtsen: use the UTC timezone when calling asciidoc.
  • #815082 on arachne-pnr by Dhole: use the C locale to format the changelog date.
  • #815192 on manpages-de by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815193 on razorqt by Reiner Herrmann: tell grep to always treat the input as text so that it works with non-UTF-8 locales.
  • #815250 on jacal by Reiner Herrmann: use the C locale to format the build date.
  • #815252 on colord by Lunar: remove extra timestamps when generating CMF and spectra and implement support for SOURCE_DATE_EPOCH.

reproducible.debian.net Two new package sets have been added: freedombox and freedombox_build-depends. (h01ger)

diffoscope development diffoscope version 49 was released on February 17th. It continues to improve handling of debug symbols for ELF files. Their content will now be compared separately to make them more readable. The search for matching debug packages is more efficient by looking only for .deb files in the same parent directory. Alongside more bug fixes, support for ICC profiles has been added, and libarchive is now also used to read metadata for ar archives.

strip-nondeterminism development Reiner Herrmann added support to normalize Gettext .mo files.

Package reviews 170 reviews have been removed, 172 added and 54 updated in the previous week. 34 new FTBFS bugs have been opened by Chris Lamb, h01ger and Reiner Herrmann. New issues added this week: lxqt_translate_desktop_binary_file_matched_under_certain_locales, timestamps_in_manpages_generated_by_autogen. Improvements to the prebuilder script: avoid ccache, skip disorderfs hook if device nodes cannot be created, compatibility with grsec trusted path execution (Reiner Herrmann), code cleanup (Esa Peuha).

Misc. Steven Chamberlain highlighted reproducibility problems due to differences in how Linux and FreeBSD handle permissions for symlinks. Some possible ways forward have been discussed on the reproducible-builds mailing list. Bernhard M. Wiedemann reported on some reproducibility tests made on OpenSuse mentioning the growing support for SOURCE_DATE_EPOCH. If you are eligible for Outreachy or Google Summer of Code, consider spending the summer working on reproducible builds!

17 February 2016

Arturo Borrero Gonz lez: An update about the HA stack on Debian


Great news! The HA stack has been finally updated and you can find now both pacemaker & corosync in Debian stretch.

This is thanks to the hard work of some people, specially Christoph Berg, Ferenc W gner, Martin Loschwitz and others.

By the time of this blogpost, in testing (stretch) you have:

Additionally, unstable contains:
which is great news. However, pcs just joined Debian and there seem to be some rough edges to be worked out.
what to do now
Now that the people mentioned above did the hard work developing the packages, please do test them and report bugs.
Having a great stretch stable release (including the HA stack) is in your hands as well.

best regards!

6 February 2016

Julien Danjou: FOSDEM 2016, recap

Last week-end, I was in Brussels, Belgium for the FOSDEM, one of the greatest open source developer conference. I was not sure to go there this year (I already skipped it in 2015), but it turned out I was requested to do a talk in the shared Lua & GNU Guile devroom. As a long time Lua user and developer, and a follower of GNU Guile for several years, the organizer asked me to run a talk that would be a link between the two languages. I've entitled my talk "How awesome ended up with Lua and not Guile" and gave it to a room full of interested users of the awesome window manager . We continued with a panel discussion entitled "The future of small languages Experience of Lua and Guile" composed of Andy Wingo, Christopher Webber, Ludovic Court s, Etiene Dalcol, Hisham Muhammaad and myself. It was a pretty interesting discussion, where both language shared their views on the state of their languages. It was a bit awkward to talk about Lua & Guile whereas most of my knowledge was year old, but it turns out many things didn't change. I hope I was able to provide interesting hindsight to both community. Finally, it was a pretty interesting FOSDEM to me, and it was a long time I didn't give talk here, so I really enjoyed it. See you next year!

Next.

Previous.